Self-organization of structures and networks 

^_ -, from merging and small-scale fluctuations. 
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Abstract 



We discuss merging-and-creation as a self-organizing process for scale- free topologies 
in networks. Three power-law classes characterized by the power-law exponents 3/2, 
2 and 5/2 are identified and the process is generalized to networks. In the network 
context the merging can be viewed as a consequence of optimization related to more 
efficient signaling. 
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c3 ' 1 Introduction 

e 

Natural processes often lead to spatially non-uniform distributions of physi- 
cal quantities. In particular scale-free structures are intriguing because they 
suggest dynamic principles that are universally applicable [1,2,3,4]. Recently 
it has been realized that many complex networks exhibit scale-free topologies 
[5,6,7]. In general, the first theoretical framework for such very skew distri- 
butions was the Simon model [8], featuring a "rich get richer" process, that 
recently has been developed into preferential attachment to explain scale-free 
networks [6]. Another approach to such broad distributions is the Self Or- 
ganized Critical models that were put forward by Per Bak and coworkers 
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[2,3,4] and were suggested for networks by [9,10]. In these models large-scale 
structures and intermittent activity emerge as a consequence of small-scale ex- 
citations of systems with inherent memory. In the present paper we elaborate 
on another class of Self-Organization models, based on the "Aggregation with 
Injection" scenario [11,12,13,14,15,16]. Interpreted as a merging-and-creation 
process, reviewing [17,18], we apply this class to complex networks and show 
that scale-free topologies emerge. The merging-and-creation process sponta- 
neously generates power-law distributions by a mechanism which is quite dif- 
ferent from the "rich get richer" scenario. As in the SOC models it is based on 
a non-equilibrium bottom up scenario, where a scale-free distribution of struc- 
tures is obtained at steady state. Thus it indeed is an appealing alternative 
to the preferential attachment models that has been suggested for growing 
networks. 



2 The merging-and-creation process 



The basic "merging-and-creation" process can be described in terms of the 
evolution of a system of many elements % = 1, 2, ...., N, each characterized by 
a scalar q^. The scalar may be either just positive [11] or both positive and 
negative [14,16]. 

As a concrete example one may think of the elements as particles and the 
scalar as the mass of a particle. Other examples are nodes of a network and 
the number of links attached to a node; companies and the financial assets of 
the company; vortices and the vorticity of a vortex and so forth. 

The prototype of the process describes a situation in which the elements in the 
system redistribute their corresponding qi according to a merging step where 
two elements % and j are chosen (typically randomly) and then are merged 
together. For illustration see Fig. lab. The merged element acquires the sum 
of the scalars % + qy We express this as 



merging : qi~^> q% + q-j 

q^O, (1) 

where the second process means that the element i is replaced by an empty 
element. This ensures that the number of elements remain constant. In par- 
allel to this, there is a creation process of elements with small \q\ ^ 0. This 
corresponds to adding a scalar q = ±1 to an empty element: 



creation: ^ = 0— *■#£ = ±1. (2) 
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Fig. 1. a) and b) illustrate the basic merging-and-creation process with b) asso- 
ciated to merging of elements with different signs, c) Refers to a merging of only 
positive elements, with creation due to injection from outside. The figure shows the 
cumulative distribution P(> q) = Ylm=q P( m ) a ^ different times from the start av- 
eraged over M = 1000-realizations for a system with N = 10 3 elements. The dashed 
curve is the exact solution that scales as P(> q) oc q l ~"' with 7 = 3/2 for large q. 
The numerical simulations are for three different times and agree with the scaling 
up to some cutoff. The deviation from the power-law above the cutoff reflects that, 
in addition, there is a single growing, large element, q oc time with P(q) = 1/N. 
The total sum Q = ^2 qi of all the other q's approaches a constant steady state 
value, d) Refers to symmetric process where both positive and negative elements 
can be merged (as the combined a) and b)). The steady state distribution P(> q), 
obtained numerically, is shown for three sizes N. The exact asymptotic solution is 
given by the dashed line (oc q^ 1 ). 

We either ensure that the average q of the system does not change by choosing 
+ 1 or —1 with equal probability in the creation step (see Fig. lab) or we 
consider the case when the average q is growing by choosing +1 every time (see 
Fig. la). Obviously there is a multitude of variants to this process, including 
for example the case where the creation event is also allowed on q 7^ elements. 
These variations of the merging-and-creation processes can be classified into 
three categories, each characterized by a unique power-law exponent for the 
distribution of scalars q among the elements. 

• Category 7 = 3/2 

The prototype process [11,12,13] can be described as 



merging 



creation : qt 



--0^q k = l- 



(3) 



Here we imagine that we have a large number of elements iV and start the 
process from all qi = 0. At each step the average scalar (q) is increased by 
1/N. Fig. la illustrates the basic process and Fig. lc shows the result from 



simulations using this initial condition (averaged over many realizations). 
As seen the probability distribution that an element has a value q, P(q), is a 
power-law with a slope that to good approximation is given by 7 = 3/2 i.e. 
P(q) oc g -3 / 2 apart from a single growing large g-element. Thus the growth 
of the average only results in the growth of a single largest element. The rest 
of the distribution is stationary and furthermore this stationary solution is 
independent of the starting condition. 

In terms of the probability distribution P the condition for a stationary 
solution in the limit of large iV is given by 

]T P( qi )P(q 2 )S qi+q2 , q - 2P(q) + S qA = 0, (4) 

which for q = gives -P(O) = and provides a recursive solution for subse- 
quent g's. As a result [11,12,13] one obtains a steady state distribution with 
asymptotic behavior P(q) oc q~ 3 ^ 2 . 
Category 7 = 2 

This corresponds to the symmetric case were P(q) = P{—q). The prototype 
for this process [14,16] is the same as for the previous case except for a 
symmetric creation: q^ = — > q^ = ±1 (see Fig. lb). This means that the 
average (q) now is unchanged during the process. Again we start with N 
empty elements (g« = 0, Vi). Fig. Id shows the result from simulations. A 
power-law distribution with 7 = 2 is generated to good approximation. 
The steady state solution in terms of P(q) is this time given by 

]T P( qi )P(q 2 )5 qi+q2 , q - 2P(q) + l[S qA + Vi] = 0. (5) 

Q1Q2 

which has the asymptotic solution P(q) oc 1/q 2 as demonstrated by Takaysu 
in Ref. [14]. In addition, the robustness of this scaling behavior is remarkable: 
If one instead of starting from a symmetric distribution with average (q) = 0, 
starts from a situation with an excess average (q) 7^ then all the excess 
((q)N) will be collected on a single large element [18]. This is similar to 
what was found for the growing case with 7 = 1.5. 
Category 7 = 5/2 x 

Here we consider the process of merging and spontaneous fluctuations among 
positive elements. Thus no negative elements are allowed, but in contrast 
to the 7 = 3/2 case the average (q) is not growing. This is achieved if at 
every merging step there is some small loss, i.e. 

q% — ► Qi + qj - 1 and qj ->■ r, 

where r G [0, 2] is a random number from a narrow distribution (r) = 1 and 
q is constrained to be > 0. The process in general corresponds to merging 
of positive elements, but also allows for spontaneous "evaporation" (when 



This process appears not to have been studied before. 





Fig. 2. a) The steady-state distribution P(> q) = Ylm=n Pi 171 ) obtained from simu- 
lations for three sizes N for the case with constrained q values (case 7 = 5/2). The 
exact asymptotic form is given by the full curve (be q~ 3 ' 2 ). The inset gives the com- 
parison between the exact solution and the simulations for the smallest q-values. b) 
The steady state distribution P(> q) for three different constraints q c = — 1, —100, 
and —N, respectively. Here the system size is N = 10 5 . Power-laws with 7 = 2.5 and 
7 = 2 correspond to the slopes of the full lines. The hgure illustrates the cross-over 
from the case 7 = 2.5 to the case 7 = 2 as the constraint on possible q values is 
relaxed. 

one q- value is zero). This process is in fact also equivalent to a number of 
other conserving processes (<& — ► qi + qj — r and qj — > r) subject to the size 
constraint that all q should be larger than some q c . 

It deals with situations where also the largest element can sometimes lose 
in the merging step, under the constraint of a lowest allowed value of q. 

With the transformation q — > q — 1 the model is equivalent to the process 



qi = qi + qj, and qj = ±1 



(6) 



This is mathematically equivalent to the symmetric process with 7 = 2 with 
the additional constraint that no element can have a scalar less than q — — 1. 
Any random choice of two elements which would lead to a merged element 
violating the constraint is abandoned and a new random choice is made. 
One notices that because the creation is symmetric with respect to q c — ±1 
the average value (q) is preserved ((g) = when starting from a symmetric 
distribution). Fig. 2a gives the result from a numerical simulation. The data 
falls on the straight line corresponding to a power-law distribution with 
7 = 2.5. The steady state solution in terms of P(q) is obtained is obtained 
in the same way as in the previous cases but the constraint changes the 
steady state condition into 



P(qi)P(q2) 
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This equation has a simple recursive solution since the q — — 1 and cases 



directly give 



P(0) = !->(-!)-' and J»(l) - P( °> P <°> 2 



4 v y 4P(-1) v y P(-l) 2P(-1) 

Eq. (7) leads to an equation in terms of g(a) = X)o° P{q)e~ aq given by 
g(a) 2 - 2g(a) - P(-l) 2 (e 2a - 2e a ) + (1 - P(-l) 2 ) cosh(a) = 0, 
which has the solution 



g(a) = 1 - yjl + P(-l) 2 (e 2 « - 2e°) - (1 - P(-l) 2 ) cosh(a). (8) 

Expanding the argument of the square root in a gives 

(3P(-l) 2 /2 - l/2)a 2 + P(-l) V. 

Now the zero order moment is just g(0) = J dqP(q) = 1 as it must. The 
second order moment must vanish by the condition (q) = 0: If 3P(— 1) 2 /2 — 
1/2 is negative then there is no solution and if it is positive then the first 



moment (q) ^ 0. So the only possibility is P(— 1) = J 1/3 which also means 
that the leading a-dependence of the square root in Eq. (8) is proportional 
to a 3 ' 2 . This in turn means that in this case the second moment diverges as 



1/q 



,2 



1 
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and it follows that the leading behavior of 7 is given by 7 — 3 = —1/2 or 
7 = 5/2. 

The exact solution can be obtained from the recursive relation starting 



with P(— 1) = y 1/3 and is plotted in Fig. 2a and its inset. Fig. 2b shows 
the cross-over from the case with 7 = 2.5 to the completely symmetric case 
with 7 = 2, by successively relaxing the constraint from q > — 1 to q > —N. 



3 Network version 



Let us now discuss the merging-and-creation process in the context of evolv- 
ing networks. The motivation for such a process in these types of complex 
systems is the gain in "simplification" that one obtains by merging nodes, 
supplemented by an overall drive to invent or excite the system by new nodes 
with new connections. The merging-and-creation scenario for networks were 
presented in Ref. [17,18], where it was shown numerically that also for networks 
the merging-and-creation gives rise to power-law distributions in parallel with 
the scalar version discussed in the present version. A simple network version 
goes as follows: 
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Fig. 3. a) Illustrates the merging move of the two random nodes and b) illus- 
trates the merging of neighbors, c) Simulation of the network version of the merg- 
ing-and-creation process for N = 2 14 and (r) = 2,4 and 8. The cumulative degree 
distribution P(> k) for the network version is shown to have a power-law distri- 
bution P(> k) oc A;' 1-7 -* with 7 — > 3/2 for N — ► 00. d) Simulation of the network 
version with the constraint of merging only of neighbors for N = 2 14 . In this case 
the power-law exponent is a function of (r) and varies from 7 = 2 to 7 = 3. 



• Choose two nodes i and j randomly. The corresponding scalar is the degree 
of the node (the number of links attached to a node). 

• The nodes i and j are merged together to a node m of degree k m = ki + kj — 
Ncommon results, with N common being the number of nodes that are neighbors 
to both i and j. These common links are deleted from the network (if i and j 
are joined by a link this is also counted as a common link). Thereby multiple 
links between pairs of nodes are removed. 

• A new node of degree k new is added to the network with the links attached 
to k new random nodes. The degree k new of the added node is a random 
number picked from a uniform distribution centered around some number 
(r). 

Fig. 3a shows that this network version of merging-and-creation gives rise to 
a power- law distribution with 7 = 3/2 ( for any (r) > 1) as expected for this 
process applied to positive quantities. 



However, a real network implementation of merging-and-creation would rather 
consists of local topological rearrangements which facilitate performance. Thus 
we consider the case where one node is constrained to be the random neigh- 
bor of the other in the merging process [17]. This would be reasonable in 
molecular networks where one protein takes over the regulatory functions of a 
neighboring protein in order to shorten the signaling pathways. With this sim- 
ple constraint on the merging-and-creation process the power-law exponent 7 
becomes a function of (r) as demonstrated in Fig. 3b, where 7 varies from 3 
to 2 with increasing (r). 
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Fig. 4. The network realization of the symmetric model, a) andh) illustrate possible 
merging moves. Positive vertices (donors) are vertices with outgoing edges and neg- 
ative (acceptors) with incoming edges, c) The cumulative probability distributions, 
N = 10 5 , for number of edges incoming or outgoing from a node. The distribution 
is scale-free P(>k) ~ l/k' 1 ^ 1 with 7 = 2. d) The cumulative probability distribu- 
tions for the changes in number of edges due to merging, Ak. The distribution is 
power-law P(>Ak) ~ Ak ~ T with exponent r = 27 — 1 = 3 from Eq. 4. 

Another interesting network application is related to merging of sun spots 
and associated magnetic field lines in the solar atmosphere [9,18]. In this case 
there are sun-spots of two polarities, and the network consists of magnetic field 
lines that make directed connections between the sun spots. In this case the 
sign (polarity) would correspond to the number of in- or out-edges [18]. Each 
vertex may have different number of edges, but at any time a given vertex 
cannot be both donor and acceptor. Further, in the direct generalization of 
the symmetric 7 = 2 case, we allow several parallel edges between any pair 
of vertices. At each time-step two vertices i and j are chosen randomly. The 
basic update is shown in panel a), b) of Fig. 4, and the result in terms of 
number of edges from any node counting multiple edges is shown in Fig. 4c. 
Also interesting in this case is the activity of events of sun spot assimilations, 
exhibiting a scaling 1/g 3 also found in the more detailed model of cascading 
magnetic loops in solar atmosphere by Hughes and Paczuski [9,10]. 

In summary, merging-and-creation opens for a new way of viewing sponta- 
neous emergence of scale-free networks, associated to systems where there is 
an ongoing tendency of simplification by merging. 
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